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FOREWORD 



The Center for Vocaiional Education c-.^ntinuing its programnnatjc R&D efforts to develop 
more effective procedures for curriculum [)lanning and design. The survey reported here represents 
one component of this long-range and multi-faceted approach. It is believed that findings of the study 
will facilitate continuing efforts of tlie profession to optimize curriculum development— hopefully, 
in new and more promising directions. 

We are indebted to the many respondents who gave of their time to provide invaluable data for 
this survey. Their cooperation and patience in rating an extensive list of activities were essential to 
the success of the effort. 

A number of Center staff members contributed over a period of time to various aspects of the 
work reported here. Earl B. Russell and Michael R. White had the major responsibility for developing 
the questionnaire and for planning and conducting the survey. Saturnino M. Ocampo, Jr., assisted in 
the data collection and in the initial drafting of results. Allen A. Wiant and Keith F. Widaman col- 
laborated in the analysis of the data and in the preparation of this report. The project was carried 
out under the direction of Frank C. Pratzner. 

We also express our thanks to Ronald C. Havelock and John R. Sanders for their reviews of the 
plans for tlie study, to Coit R. Butler for liis critique of the data collection instrument, and to Nevin 
R. Frantz, Jr. for his review of this final report. 



Robert E. Taylor 
Director 

The Center for Vocational Education 
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ABSTRACT 



This study was conducted to enhance basic knowledge concerning the activities and needs of 
curriculum developers, with emphasis on those relating to vocational education, to enable research 
to be more responsive to their concerns. More than 300 persons actively involved in curriculum 
development in public education, business and industry, and government responded to a question- 
naire based upon a list of sixty-eight curriculum development activities in five categories: (a) cur- 
riculum management and administration, (b) selection and organization of content, (c) selection and 
organization of materials, (d) design of instructional plan and alternative strategies, and (e) evaluation 
of curiiculum. Ratings were given for the degree of problem/need encountered in the performance of 
each activity, and the relative importance of each to the respondent's job. in addition, background 
data were obtained from each respondent. 

In analyzing the data, the importance and problem/need ratings provided by respondents were 
combined to produce a summary "criticality" score for eacfi activity. For the respondent group 
taken as a whole, the most critical activities were found to be in the categories of curriculum man- 
agement and administration, content selection and organization, and evaluation. Eight groups of 
related activities were also identified, with differing criticalities for curriculum developers in local, 
?tate, and national R&D lab situations. Those working at the local level were most critically con- 
cerned about the seler.rion and organization of content, the exchange of information to facilitate 
our/inulum irr/)r;;vement, and curriculum evaluation. The use of job analysis in curriculum content 
denvcuion, a • d the integration and adaptation of curriculum accomplishments elsewhere, were more 
critical to those in state agency contexts. Those in national R&D labs were more concerned than all 
otiiers with problems associated with the design of learning experiences and instructional strategies. 
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ACTIVITIES, PROBLEMS AND NEEDS OF 
CURRICULUM DEVELOPERS: A NATIONAL SURVEY 



PROBLEM AND OBJECTIVES 



Curriculum developers are a diverse group, engaging in activities on college campuses, in state 
departments of educa.ion, research coordinating units, local schools, private agencies, industry, and 
in government agencies such as military technical training schools. However, basic i:^.Gwledgc about 
the activities and needs of curriculum developers is grossly incomplete (Coney et a!., 19G8). Without 
this basic knowledge, individual researchers and agencies that serve curriculum deveiopers cannot 
focus their effort to provide the most needed curriculum development tools. Therefore, a system- 
atic effort was undertaken to identify more clearly the population of curriculum developers and to 
determine their problems and needs. In keeping with The Center's overall mission, the emphasis of 
the study was upon problems and needs germane to vocational education. 

The study had two objectives: 

1. To identify curriculum developers in the three general areas of public education, business/ 
industry, and government, and to obtain data on general characteristics of these groups, including 
types of employing institution, target population(s) for which curricula are developed, and the pro- 
portion of time devoted by curriculum developers to curriculum development activities. 

2. To identify curriculum developers' activities, problems, and needs associated with (a) cur- 
riculum management and administration, (b) selection and organization of content, (c) selection and 
organization of materials, (d) de:iign of instructional plan and alternative teaching strategies, and 

(e) evaluation of curriculum. For the purposes of this study, curriculum development was defined 
as a total process incor[3orating these five categories of activity. 
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DESIGN AND METHODOLOGY 



Population and Sample 

The target population of the study was vocational curriculum developers in public education, 
business/industry, and government. These three sectors of the population were divided into various 
strata and further sub-categorized as shown in Table 1 . 

Due to the size and diversity of the population, several sampling procedures were needed. To 
obtain a sample as nearly representative of the entire population as possible, a stratified sampling pro- 
cedure was used, based upon institutions engaged in curriculum development. Within each stratum, 
a sample of institutions was selected by simple random sampling, purposive sampling, or a combina- 
tion of random and purposive sampling. The criteria utilized in purposive selections were: 

1. Knowledge on the part of the project staff ^hat the activities of the institution to be repre- 
sented were reasonably comprehensive of ali curriculum development activities, or that the institu- 
tion to be represented (manufacturer, trade union, etc.) was involved with occupational activities 
related to those taught in public school vocational programs. 

2. A knowledge of individual respondents within institutions, often considered in conjunction 
with the preceding criterion. Individuals were sought with whom project staff members had prior 
communication, or who were known through their work in the curriculum field and their publications 
in the field of curriculum development. 

3. Geographical distribution. Representation from all states having technical institutes and 
area vocational centers was sought. Institutions were selected from those providing comprehensive 
program offerings. In general, these were selected in proportion to the size of the population served. 

4. Size of institution being represented, indexed by size of staff. Larger institutions were 
selected in several instances on the assumption that they had greater staff specialization and more 
resources available to engage in systematic curriculum development activity. 

In cases where the comprehensiveness of the activities of the institutions in a sampling category 
was not known, a probability sampling technique was used, giving each institution an equal, chance 
of being included in the study. Although a probability sampling technique is usually considered the 
best way to ensure a representative sample, purposive sampling was used in many sampling categories 
to guarantee the inclusion of those institutions engaged in comprehensive curriculum development 
activity. 

Lists of curriculum developers were taken from computerized mailing lists of The Center for 
Vocational Education; National Business Lists, Directory of Accredited Private Trade and 
Technica/ Scfiools (1974); and directories of the U.S. Department of Health, Education and Welfare, 
U.S. Department of Labor, U.S. Department of Transportation, and the Civil Service Commission. 



Table 1 



Taxonomy of Curriculum Developers, Sample Sizes, 
and Questionnaire Return Rates 



Population 


Agencies/ Institutions 
(Sampling Unit) 


Sample Size 


Usable Returns 


Sector 


Stfata 


Design Actual 


No. Percent Percent 
of Sample of Total 


Public 
Educaiion 
(n 370. 

53".'. of 
Sijrnple) 


National 


R&D Laboratories and Centers (50"i) pur- 
posive sample, criteria 1 and 7) 


10 20 


d 40 2 


State 


Post-Secondary 

a. Community-Junior Colleges 

b. Technical Institutes 

c. Area Vocational Schools 

d. Vocational Departments In Colleiies 
& Universities 

(pu'posive & proportional sampling, criterion 3) 

State Divisions of Vocational Education 
in State Departments of Education (50% 

niirr^n^iv/p ^nmnlp rritPrin 1 nnH 4) 

State Instructional Materials Laboratories 
(50"f') purposive sample, criteria 1 and 4) 

Research Coordinating Units in Vocational Edu- 
cation (50% purposive sample, criteria 1 and 4) 


60 59 
22 22 
86 86 
32 39 

25 68^ 
10 10 

26 22 


37 63 1 1 
13 59 4 
57 66 17 
29 74 9 

38 ^6 12 
9 90 3 

9 41 3 


Local 


Local School Districts (2% random sample of 
school districts with secondary enrollments 
of 2,000 students or more) 


100 102 


60 59 18 


Business/ 
Industry 
{n-20l; ■ 
29% of 
Sample) 


Pre-Job 


Private Trade and Techriical Schools (20% 
purposive sample, criterion 1 ) 


100 99 


28 28 9 


Large Manufacturing Firms— Pre/on Job 
Training v,/ith, 1,000 or More Employees 
•*«»^5% randi^m sample) 


66 66 


12 20 4 


On-Job 


Trade Organizations/Associations (pur- 
posive sample, criterion 1) 


25 25 


0 0 0 


Other 


Private, Non-Profit, Behavioral R&D 
Agencies (purposive sample, criteria 1 and 2) 


10 P 




Governnnent 
(n - 100; 
15% of 
Sample) 


National 


Military Agencies, i.e.. Technical Schools, 
Training Commands, Personnel Labs o* 
the USAF, Navy, and the Army 
(purposive sample, criterion 1) 

Intra-Agency Training Programs Dept. of 
Labor, Dept. of Transportall(»n, Civil 
Service Commission, Htr. etc. our- 
poslve sample, criter»a l ..nd 4) 

Other Manpowoi ?^:ogr:,r7is (pur 
poslve sample, criterion 1) 


25 35 
50 


13 50 4 

14 40 4 


Miscellaneous 
(n - 77:3% 
of Sample) 




Specific Re^'erriih Obtained in Progress 
of the Stud', 


22 14° 




Totals 


693 693 


328 47 100 



Includes manpower programs 



b - Included with R&D Labs and Centers 
c - Distributed in other categories 
d - Unused; remainder distributed 
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Questionnaires were sent to a total sample of 693 curriculum developers. Respondents who 
had not returned questionnaires within one month of the initial mailing were sent a second copy of 
the questionnaire along with a letter stressing the importance of the study and the information they 
could provide. One week later, a telephone follow-up was conducted of a random sample of the non- 
respordents to obtain more returns and identify reasons for tlieir failure to return the questionnaires. 
As a result of follow-up efforts, the total number of returned questionnaires was 373, or 54 percent 
of the total samp e. Data collection was conducted during October and November, 1974. 

The returned qup'Stionnaires were reviewed for completeness, legibility, and appropriateness of 
response. A questionnaire was considered usable if no more then seven of the 136 requested ratings 
(5 percent) were missing. Using this criterion, 328 questionnaires were classified as usable, or 47 
percent of the total sample. The distribution of usable responses is shown in T?b!e 1. 

Table 1 also contains both "Design" and "Actual" sample sizes. These sample sizes correspond 
to the designed number of respondents in each category and the number classifiable as belonging in 
each category after returns were received, respectively. Respondents who did not reiurn question- 
naires were assumed to have been in the assigned sample category. Respondents who did return 
questionnaires were assigned to the category to which they indicated they belonged, and this resulted 
in an alteration of the actual sample sizes per sampling unit. 



Instrument 

The survey questionnaire contained two sections. The first section was designed to obtain back- 
ground information on each respondent and institution represented. Included were such things as 
the primary audiences of their efforts, number of years of training, and the number of years spent 
in curriculum development. 

The second section of the questionnaire contained sixty-eight curriculum development activities 
in the areas of (a) curriculum management and administration, (b) selectiori'and organization of con- 
tent, (c) selection and organization of materials, (d) design of instructional plan and alternative teach- 
ing strategies, and (e) evaluation of curriculum. Each of the sixty-sight activity statements was rated 
on two scales. The first was a O-to-3 scale which yielded responses indicating whether the respondent 
performed the task and, if so, the degree of problem encountered in performing the task. The second 
scale was a 1-to-5 scale to indicate the importance of the activity to the respondent's joh. Definitions 
of the scale intervals are given in Appendix A. 

A reviev" panel made up of nine curriculum developers in the Columbus, Ohio, arej in the educa- 
tion, business/industry, and government sectors critically reviewed the survey instrument to ePfoure 
that the task statements were understandable and adequately comprehensive of all curriculum develop- 
ment activity. This effort was an explicit attempt to ensure the content validity of the instrument. 
The questionnaire was revised on the basis of the panel's recommendations. 



Data Analysis 

Background data were separately summarized and analyzed. Differences in the background 
characteristics among the three primary groups of curriculum developers were assessed using the 
chi-square statistic. 
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Summary statistics (e.g., means and variances) were computed for eacli activity from ratings 
given by all respondents. Statistics were computed separately for the Problem/Noed and Importance 
scales. Missing data were estimated based upon the mean of the ratings obtained from other respond- 
ents. An r^'ternative way to have analyzed the data from the Problem/Need scale would have been 
to com; jII summary statistics on data supplied by performers of each task rather than across all 
responu.-nts. The "performers only" method would have resulted in statistics which accurately re- 
flected the data for those who performed each activity, but would not have represented the ratings 
of the "average" curriculum developer. The "performers Oiily' method would also have resulted in 
statistics which would not have been comparable across activities, would have precluded the formula- 
tion of criticality rating scores, and would have precluded a separate factor analysis of the Problem/ 
Need ratings. For these reasons, the "performers only" method of analysis was not followed. How- 
ever, the mean ratings for performers only in the three major groups of curriculum developers may be 
calculated from the data provided in Appendix A. 

Factor analysis was the method chosen to reduce the activity performance data. In addition, 
the desire of the project staff for the identification of critical activities of curriculum developers led 
to the formulation of a single "criticality" rating which consolidated the Problem/Need and Impor- 
tance ratings. Accordingly, activities highest on this criticality index were those which were rated 
as being both very important and involving great difficulty. Activities which were very important 
but routinely performed, or which presented a great problem but were relatively unimportant, ac- 
quired moderate criticality ratings. Finally, those activities which were viewed as neither important 
nor a problem to perform earned low criticality ratirgs. The criticality ratings thus provided an in- 
dex of the need for focusing future attention and effort on certain of the activities performed by 
curriculum developers. 

The factor analysis was conducted in two stages, employing the criticality scores for the sixty- 
eight tasks to produce eight rather clearly defined primary factors and one general factor. Factor 
scores were then computed for each respondent on each factor by the least-squares method (Tucker, 
1971). Separate one-way analyses of variance were performed on the factor scores for each factor, 
with group membership serving ss the levels of the factor in the analys3S of variance. Table 1 lists 
thi'teen categories of curriculum developers with usable returns, and these thirteen categories served 
as the levels for group membersliip. Where a significant analysis of variance was found, post-hoc com- 
parisons, using Dunn's +pst, were made between the three major groups of respondents- education, 
business/industry, and government— and between various subgroups within the education group 
(Kirk, 1968). 

More detailed explanations of the reasons for the calculation of Problem/Need values as stated, 
of the method used to formulate criticality listings, and of the factor analysis procedure, are given 
in Appendix C. 
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DISCUSSION OF RESULTS 



As shown in Table 1, response rates varied considerably for the various subsamples, those for 
business/industry and government in particular being lower than desired. Consequently, caution 
should be observed in generalizing the findings for these groups. 



Data on Respondents 

Background information obtained from respondents showed them to possess an average expe- 
rience of fifteei. years in teaching or training and nine years in the development of curriculum and 
instruction; howaver, the data did not provide for deitnmininp whether these types Df experience were 
obtained consecutively or concurrently. Also, the lengths of the experience of individuals within the 
sample varied greatly. 

Respondent groups also differed with regard to audiences being served, based upon their selec- 
tions from a list of eleven audiences provided in the questionnaire. Table 2 summarizes the responses 
of those in public education, business, and government, (totalling 100 percent for each respondent 
group) with regard to their respective audiences. Audiences are shown arranged into four broad 
groupings (vocational/technical education, general education, industry training, and government 
training) for which sub-totals are also given. Thus, as may be noted, clients in vocational/tech ,ical 
and general education together comprised 90.9 percent of the audiences of the curriculum developers 
in public education. Their largest single group of audiences was the vocational/technical group (52 
percent), and their largest single audience within this group was post-secondary vocational/technical 
education (21 percent). 

Curriculum developers in industry serve h wide range of clients, and in common with those in 
public education, their largest audience was in vocational/technical education (44 percent), followed 
by those concerned with industry training (26 percent). The primary audience of curriculum devel- 
opers in both public education and in business/industry was post-secondary vocational/technical edu- 
cation. However, the indication was that post-secondary vocational/technical education was more 
important to curriculum developers in business and industry than to those in public education; one 
of three respondents from industry selected this as a primary audience, whereas only one of five 
from public education so indicated. 

Government training programs, although clearly the primary audiences of the curriculum devel- 
opers in the government sector, were closely followed by audiences in general education. In fact, 
audiences in general education were indicated to be nearly as important to curriculum developers in 
government as to their counterparts in the public education sector. Government respondents were 
less involved with audiences in vocational/technical education, but college and university audiences 
were indicated to be of greater importance to them than to respondents in either the public educa- 
tion or industry groups. 

Those addressed in the survey wero also asked to provide an estimate of their involvement in 
curriculum development, expressed as a percentage of their total work schedule, where curriculum 
development was defined as being inclusive of the five broad categories of activities previously noted. 
Their estimates averaged as follows: 

7 
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Table 2 



Audiences of Curriculum Developers in 
Public Education, Business, and Governmenl 



Public Business/ 
Education Industry Government 

Audiences (%) (%) (%) 



Vocational/technical education 



Post high school (vocational/technical) 

High school (vocational) 

All levels of vocational education 

Sub-total 

General education 

College/university 

Post high school (general) 

High school (general) 

Middle, or junior high school 

K-6 elementary 

All levels 

Sub total 

Industry training programs 

Government ^raining programs 



20.7 32.9 12.0 

19.4 7.1 4.0 

12.0 4.3 4.0 

52.1 44.3 20.0 



6.4 5,7 12.0 

3.2 7.1 4.0 

9.2 2,9 10.0 

8.3 1.4 6.0 

6.4 - 4.0 
5.3 1.4 

38.8 18.5 36.0 

4.1 25.8 2.0 

5.1 11.4 42.0 
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Respondent Group 



Percent of Time in Curriculum Development 



Public Education 48.0 

Business/Industry 30.1 

Government 48.1 

A majority of the respondents in business/industry reported devoting 25 percent or less of their time 
to curriculum development; none were so engaged on a full-time basis. This finding was in contrast 
to the other two groups. A substantial number from both the public education and government 
groups reported devoting at least 75 percent of their time to curriculum development activity. 

An estimated categorical distribution of each respondent's curriculum development effort was 
also requested. Estimates were to total 100 percent for each respondent, regardless of total time 
spent in curriculum development. The estimates for the total sample are summarized in Table 3. 

Table 3 

Distribution of Curriculum Development Effort 

Activity Category Average Percent of Time Spent 

Curriculum management and administration 42.7 

Selection and organization of content 15.3 

Selection and organization of materials 13.7 

Design of instructional plan/alternative teaching strategies 12.3 

Curriculum evaluation 16.0 



Total curriculum development effort 100.0 



Although the overall average time spent in curriculum development differed for the three groups of 
respondents, their respective distributions of that time were found to be very similar and adequately 
represented for each group by the data in Table 3. Chi-square tests of significance were performed 
on the distributions of the three groups, and none of the differences between them were found signi- 
ficant at the .05 level. 



Activity Ratings 

Ratings provided a large base of data reflective of the Importance and Problem/Need of each of 
the sixty-eight activities listed in the questionnaire. In addition, these ratings gave support to the 
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content validity of the list of activities U:>9d to describe curriculum development. Sixty-five of the 
sixty-eight activities in the inve ntory were reported as being performed by more than half of all the 
respondents. Thus, the inventory itself comprises a useful resource for a competency based approach 
to preparing curriculum developers. Appendix A contains the list of activities, percentages of respon- 
dents performing each activity, and summary ratings of Importance and Problem/Need. 

Activities rated high on both Importance and Problem/Need by all respondent groups tended 
to lie in the areas of curriculum management and administration, content selection and organizatioi . 
and evaluation (apriori categories of activities utilized in the questionnaire). Conversely, activities 
involving curriculum materials selection r^nd organization, and the design of instructional plans and 
strategies (the remaining categories) were given lower ratings on both dimensions. V\/ithin the more 
highly rated activity categories, however, activity rankings differed from group to group; and respon- 
dent subgroups indicated by their Importance and Problem/Need ratings performance concerns which 
did not follow those of the respondent group as a whoie. The highly rated activities of various re- 
spondent subgroups are compared in Appendix B. Howe-'er, because of the two dimensions rated and 
the complexity of the sample, such comparisons were difficult because they involved piecemeal con- 
sideration of the data. Syctomatic and comprehensive analysis required more powerful data reduction 
methods, as discussed in tho- following paragraphs. 



Most Critical Activities 

The criticality score for each activity, derived from the Problem/Need and Importance :'atings, 
provides a composite index of importance and the nyed fo'' further research, deve'opmer. and 
training or in-service effort. Although the criticality scores do not identify the type of assistance 
needed, many of the activity statements themselves provide this indication. 

Criticality scores were highest for those activities rated highest with respect to both Problem/Need 
and Importance. Activities of considerable importance but routine in performance, as well as those 
characterized as highly problematic but of limited importance to curriculum developers' jobs, had 
lower criticality scores. The twenty "most critical" activities, as defined for the responc''?nt group 
as a whole, are listed in Table 4 in rank order of their mean criticality scores (actual scores are given 
in Appendix D). The table also indicates the criticality rank of each of the twenty activities for the 
public education, business, and government respondent groups respectively. Numerals in parentheses 
following each activity statement give the activity's numerical identification, which permits reference 
to the activity data in Appendix A. 

Although the most critical activities of each major respondent group tended to be among those 
listed in Table 4, there were some notable exceptions. The fourth most critical activity of the busi- 
ness/industry group, for example, was not found among the twenty most critical activities of the 
overall group. For this reason, and to give separate rank ordered lists for respondents in public educa- 
tion, business, industry, and government. Table 5, 6, and 7 are provided. Table 4, in particular, assisted 
in making comparisons between the groups. For example, ten of the twenty activities listed were 
among the twenty most critical for each group (identified by asterisks). Further, it was observed 
(by comparing their activity numbers with the activity list in Appendix A) that these ten activities 
were distributed with respect to the apriori categories employed in the questionnaire as follows: 

Curriculum management and administration 3 

Content selection and organization 4 

1 (* 

Materials selection and organization 0 
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Table 4 

Twenty Most Critical Activities for All Respondent Groups 



Rankings 



Activity 




* Deterr ine the knowledge/concepts to be includes in a curriculum 

\ IN u . ou/ 


1 


3 


2 


1 


* Evaluate the appropriateness of existing curriculum - )'t:nt 'or 

inc liiLcilUcU puipuac Ul Icailllliy u*t; 


2 


2 


1 


4 


* notormint* tKo nprfnrmanrp nhiprtiup*: tn hp inrliiripn' \t. ClITf 'f.'Ulum 

L/C LCI llllllC LlIC ^Cl lUllllailOC VJ UJCU ll VCO LU U\Z ll I ^ l UUCv« IFt C (vUlH'OUIUIII 

(No. 34) 


3 


8 


3 


2 


Prpnarp a Innn rannp nl;)n fnr n irrin 1 li im /inQtri irt innal nrnnram 

development (No. 24) 


4 


1 


- 


15 
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terms (No. 17) 


5 


5 


13 


6 


loentiTy new metnoos ano proceoures lor uevciuping curriculum 
and instructional materials (No. 21) 


6 


6 


14 


5 


*State the ultimate performance objectives (No. 39) 


7 


12 


6 


3 


* Determine which curriculum will be developed (No. 5) 


3 


g 


8 


8 


* Compare actual learning outcomes with intended learning outcomes 
(No. b/) 


q 


10 


5 


11 


Determine the attitudes to be developed in a curriculum (No. 6) 


10 


11 


11 




Determine how new curricula are going to require instructors to 
change or be retrained (No. 60) 


11 


4 


- 


- 


* Revise curriculum and instructional materials based on field 
test data (No. 68) 


12 


14 


9 


IS 


Formulate policies and priorities for curriculum planning and 
development (No. 23) 


13 


16 


- 


7 


Conduct staff in-service training for curriculum revision efforts (No. 1) 


1 1 


7 






Determine where the same curriculum problems are being worked 
on elsewhere (No. 16) 


15 


13 






* State the requisite enabling objectives (behavioral requirements: 
skills, knowledge, attitudes) (No. 40) 


16 


19 


17 


10 


Utilize experienced instructors to identify curriculum content 
and to write instructional materials (No. 28) 


17 


18 


7 




Recommend instructional techniques and learning experiences 
as part of a curriculum plan (No. 54) 


18 




10 


14 


Evaluate the utility of student performance und enabling 
objectives (No. 62) 


19 




12 




Identify alter-^^^tive methods of instruction (including major resources 
and types of <structional aids) (No. 61 ) 


20 


20 




17 



"Ranked among the twenty most critical by each group. 
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Tables 

Ten Most Critical Activities for the Public Education Group 



Activity 

Rank Activity Number 

1 Prepare a long range plan for curriculum/instructional program development 24 

2 Evaluate the appropriateness of existing curriculum content for the intended purpose 

of learning g4 

3 Determine the knowledges/concepts to be included in a curriculum 35 

4 Determine how new curricula are going to require instructors to change or be retrained 60 

5 Define curriculum development goals or objectives in operational terms 17 

6 Identify new methods and procedures for developing curriculum and instructional materials 21 

7 Conduct staff in-service training for curriculum revision efforts 1 

8 Determine the performance objectives to be included in a curriculum 34 

9 Determine which curriculum will be developed 5 

10 Compare actual learning outcomes with intended learning outcomes 67 



Tables 

Ten Most Critical Activities for the Business/Industry Group 



Rank 



Activity 

^c^'vity Number 



1 Evaluate the appropriateness of existing curriculum content for the intended purpose of learning 64 

2 Determine the knowledges/concepts to be included in a curriculum 35 

3 Determine the performance objectives to be included in a curriculum 34 

4 Survey former students to assess curriculum effectiveness g5 

5 Compare actual learning outcomes with intended learning outcomes 57 

6 State the ultimate performance objectives 39 

7 Utilize experienced instructors to identify curriculum content and write instructional materials 28 

8 Determine which curriculum will be developed 5 

9 Revise curriculum and instructional materials oased on field test data 68 

10 Recommend instructional techniques and learning experiences as part of a curriculum plan 54 



Table 7 

Ten Most Critical Activities for the Government Group 



Activity 

Rank Activity Number 

1 Determine the knowledges/concepts to be included in a curriculum 35 

2 Determine the performance objectives to be included in a curriculum 34 

3 State the ultimate performance objectives 39 

4 Evaluate the appropriateness of existing curriculum content for the intended purpose of learning 64 

5 Identify new methods and procedures for developing curriculum and instructional materials 21 

6 Define curriculum development goals or objectives in operational terms 17 

7 Formulate policies and priorities for curriculum planning and development 23 

8 Determine which curriculum will be developed 5 

9 Assess the adequacy of existing curriculum and instructional materials for use in local curriculum 
development 52 

10 State the requisite enabling objectives (behavioral requirements: knowledge, skills, attitudes) 40 



Design of instructional plan/strategies 



0 



Curriculum evaluation 3 

Activities of unique concern to any one group of respondents were also readily identifiable. For 
exannple, the most critical activity of the public education group involved the preparation of long 
range plans for curriculum development, which was an activity of much less concern to the other 
two groups. As discussed in the following section, all activities tended to be seen as moderately crit- 
ical. Hence, these tables serve to highlight the unique needs of each group in terms of their most 
critical activities. 



Factor Analysis of Criticality Scores 

Factor analysis was performed to further reduce the large quantity of data obtained to more 
manageable and interpretable dimensions. This involved the empirical grouping or clustering of crit- 
ical activities, and enabled comparisons to be made between respondent subgroups on the basis of 
these clusters (factors). The factor analysis resulted in the identification of nine factors, one general 
and eight primary factors.^ The primary factor loadings (loadings > .245) are listed in Table 8; fac- 
tor loadings for the general factor are given in Table 15 (Appendix E).^ Inspection of the activities 
contained in each of the primary factors (Table 8) showed the tendency for each factor to be domi- 
nated by activities which were originally grouped together in a single category, indicating a strong 
relationship between the apriori categories and the results of the factor analysic>. 

The nine one-way analyses of variance performed on the factor scores^ revealed seven significant 
.:vero1 differences. The analysis of the general factor was significant (p < .01), while analyses of 
PrifTiary Factors 1, 2, 3, 4, 5, and 8 were highly significant (p < .0001). The analyses of Primary 
Fdr;tors 6 and 7 revealed no significant overall effects. 

The group means on the factors were plotted and visually inspected to identify major trends. 
The two subgroups in business were treated as one combined group for the post hoc analyses since 
they tended to score in a similar manner on all factors; the same was true of the two government 
subgroups. Among subgroups in education, four subgroups— including curriculum developers in 



Both the scree test and interpretability indicated the presence of eight factors in the final first- 
order factor analytic solution. The eight factors explained 86 percent of the estimated common 
variance and 55 percent of the total variance. The binormamin rotation yielded the solution with 
the best simple structure. 

The second-order factor analysis indicated the presence of only one factor. This factor accounted 
for 98 percent of the second-order estimated common variance and 45 percent of the second-order 
total variance. The primary factor pattern matrix, after the Schmid-Leiman transformation, is given 
in Appendix E. Activities there are listed in numerical order; they may be identified from their nu- 
merical designation using Appendix A. 

^The factor reliability of the general factor was .97; the reliabilities of the eight primary factors 
ranged from .60 to .39, with a median of .48. These reliability estimates are lower bound estimates, 
and test-retest reliabilities would tend to be higher because of the lack of overlap of the variables. 

^See "Data Analysis." 
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Tables 
Critical ity Primary Factors 



f Factor 
Category loading 



Factor!, Identify Content Objective:. 

39 State the ultimate performance objectives B .595 

35 Determine the l(nowledges/concepts to be included in a curriculum B ,590 
34 Determine the performance objectives to be included in a curriculum B .575 

37 Select curriculum content as intended learning outcomes B .522 

40 State the requisite enabling objectives (behavioral requirements: knowledge, skills, attitudes) B .506 

36 Identify prerequisite and enabling skills and knowledges for each curriculum or instructional unit 8 ,.504 

38 Sequence the series of intended learning outcomes B ,455 

41 Identify potential units of instruction B .379' 
1 7 Define curriculum development goals or objectives in operational terms A .337 
13 Describe and promote curriculum activities and products at local, state, regional, or national 

0) professional meetings A -.301 

4^ Prepare statements of student performance objectives from occupational task performance data B .248 

Factor 2, Design Learning Experiences 

58 Df termine new !Ole relationships between instructors and pupils required by new curricula D ,531 

59 0..n?rmine how demanding new curricula will be of s*udents D .479 
57 Cieate situations by which students can exhibit intended learning outcomes D .440 
56 Design alternative instructional treatments (i.e., materials and experiences) for variously defined 

subgroups of students q 430 

60 Determine how new curricula are going to require instructors to change or be retrained D .418 

61 Identify alternative methods of instruction (including major resources and types of instructional aids) D .399 
53 Derive instructional strategies and learning experiences to support specific learning outcomes D ,31 7 

,68 Revise curriculum and instructional materials based on field test datj E ,284 

43 Determine the extent that proposed curricula have gone through experimentation or validation B .281 

62 Evaluate the utility of student performance and enabling objectives E ,267 
50 Survey students to identify individual and group characteristics D .254 

25 / 26 

o 

ERIC 



Factor 3. Communicate and Gather Information 



10 


Provide information to the community about the nature and content of curriculum 


A 


.487 


1 


Conduct staff in-service training for curriculum revision efforts 


A 


,413 


4 


Survey employers to identify sl<ills and l<nowledge needed by students for entering employment 


A 


,366 


3 


Survey instructors formally to identify their needs for curriculum/instructional materials 


A 


.356 


5 


Determine which curriculum will be developed 


A 


.326 


9 


Obtain and analyze survey data (e.g., censui, community needs, company needs) to help establish 








curriculum policy and development priorities 


A 


,310 


7 


Consult with employers to identify near-future trends and developments in occupations as they 








relate to curricula 


A 


.304 
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D 
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29 


Prepare occupational task inventories for use in curriculum development 


B 


.490 


27 


Survey employees in business/industry about their job tasks 


B 


.480 


30 


Employ trained job analysts to obtain information on worker pf?rformance for use in curriculum 








development 


B 


.461 


7 


Consult with employers to identify near-future trends and developments in occupations as they 








relate to curricula 


A 


.429 


32 


Visit job sites to record observations of worker performance for use in curriculum development 


B 


.417 


4 


Survey employers to identify skills and knowledge needed by students for entering employment 


A 


.386 
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36 


Survey employers of former students to assess curriculum effectiveness 


E 


.649 


65 


Survey former students to assess curriculum effectiveness 


E 


.614 


67 


Compare actual learning outcomes with intended learning outcomes 


E 


.432 


64 


Evaluate the appropriateness of existing curric\ilum content for the intended purpose of learning 


E 


.384 


51 


Use standard tests to identify individual and group characteristics 


D 


.258 


50 


Survey students to identify individual and group characteristics 


0 


.253 



2*1 (continued) 
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Activity^ Factor 

^^"^^"y Category Loading 

f^actor 6. Select and Organize Materials 

47 Prepare teaching-learning materials (e.g., transparencies, inforination slieets, models, slides) C ,516 
49 Develop total instructional delivery systems (e.g., instructional packages, audiovisual materials, etc.) C .41 1 

48 Describe the physical facilities and list necessary equipment, tools, and/or materials as part of a 

curriculum plan C ,410 

46 Prepare specifications for development of necessary curriculum materials and instructional aids C ,362 
12 Vi/rite materials (e.g., brochures, journal articles) to promote developed curriculum and instructional 

materials A .313 

1 1 Supervise design of exhibits and displays to promote developed curriculum and instructional materials A .294 

31 Search published textbooks as a primary source of content for curriculum B .284 

45 Adapt available curriculum and instructional materials to meet local curriculum development needs C .266 

Factor 7. Formulate Curriculum Policy and Plans 

24 Prepare a long-range plan for curriculum/instructional program development A .61 1 

25 Prepare an annual plan for curriculum/instructional program development A .573 
23 Formulate policies and priorities for curriculum planning and development A .510 

15 Determine the developmental and operational costs of new curricula/instructional programs A .305 

20 Use computerized information systems to obtain data for deci5ion*making A .272 
5 Determine which curriculum will be developed A .271 

26 Supervise curriculum and instructional materials development work A .264 

Factor 8, Maintain Awareness of Relevant Developments Elsewhere 

1 6 Determine where the same curriculum problems are being worked on elsewhere A ,473 
19 Relate curriculum information derived elsewhere to a local problem A .414 
14 Work with local personnel to modify and develop support materials for local use of curriculum 

and instructional materials A ,359 

42 Determine the content and success of new curriculum developed elsewhere B »346 
22 Cooperate with other agencies to jointly develop curriculum and instructional materials A ,31 1 

21 Identify new methods and procedures for developing curriculum and instructional materials A .309 

43 Determine the extent that proposed curricula have gone through experimentation or validation B ,281 
33 Utilize advisory committees to assist in curriculum formulation B ,248 



^ A: Curriculum Management and Administration 

B: Selection and Organization of Content 

C: Selection and Organization of Materials 

D : Design of Instructional Plan and Alternative Strategies 

E : Evaluation of Curriculum 
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community and junior colleges, technical institutes, area vocational schools, and local school districts- 
tended to score in similar fashion on all factors; these four subgroups were combined for post hoc anal- 
yses and, reflecting their places of employment, were labelled the Local Education Agency (LEA) 
group. Four other subgroups in educatiCd—state departments of vocational education, state instruc- 
tional materials labs, research coordinating units, and vocational education departments in colleges 
and universities— showed similar score profiles across all factors; these groups were combined for post 
hoc analyses and labelled the State Education Agency (SEA) group. The one remaining education 
subgroup was the national R&D labs (NRD); this subgroup was combined with the SEA group for 
some comparisons, although differences between SEA and NRD were carefully noted. The summary 
of post hoc comparisons among groups is given in Table 9. 

General Factor I: General Rated Criticality . 

All items loaded moderately or highly on Factor I, indicating that curriculum developers tended 
to judge all activities on the instrument as critical to their jobs in much the same way. That is, if a 
curriculum developer judged one activity as critical to his/her job, then he/she tended to regard the 
sixty-seven other activities as critical also. 

The content interpretation of this general factor was that it indicated a general criticality of all 
the curriculum development activities. However, the appearance of the general factor could have been 
due to response bias, some respondents rating all tasks generally higher and others rating tasks generally 
lower. At present there is no way to decide between the two interpretations, and both may be partially 
correct. Only further research using independent criterion measures could begin to establish the cor- 
rect meaning of the general factor. 

As shown in Table 9, the only significant differences among the groups occurred when the cur- 
riculum developers in education, considered as a single group, were compared to those in business and 
government. The education group regarded all activities, as a whole, as more critical to their jobs than 
did the business and government groups; the business and government groups did not differ significantly 
in their rated criticality of all the activities to their jobs. 

Primary Factor 1: Identify Content Objectives 

Primary Factor 1 was found to contain items which together describe a curriculum content 
selection process. The process thus described includes the identification of prerequisite and enabling 
skills and knowledges, and the knowledges and performance objectives to be included in the curric- 
ulum. It also includes the statement of the above outiined skills and knowledges, especially in terms 
of performance objectives where appropriate. Finally, it deals with the formulation of larger units 
of instruction and the sequencing of those larger units into a meaningful series of learning outcomes. 
Overall, the process as described by activities in the factor involves the determination and statement 
of prerequisite and to-be-learned content. 

Since this factor dealt with the identification of curriculum content for use in an instructional 
program, it was expected that curriculum developers closest to the scene of instruction would regard 
these tasks as more critical to their jobs, and thus would score higher on the factor. In line with this 
expectation, curriculum developers in business and in government did score significantly higher than 
their counterparts in education, taken as a whole. However, within the education group, the LEA 
group exhibited higher scores than both the SEA and the combined SEA-NRD groups. The overall 
outcome was that the local agency groups scored uniformly higher than the state-level groups on this 
factor. 
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Table 

Post-Hoc Comparisons Between Groups of Curriculufil'Devglopers on Criticality Factors 

Comparison^ 

Fsctor ' — ■ ' "■ — — 

EdvsBus&Gov EdvsBus EdvsGov Bus vs Gov LEA vs SEA LEAvsSEA + NRD SEAvsNRD 

General Factor: 
I: General Rated 

Criticality Ed>(Bus+Gov)** Ed>Bus** Ed>Gov** 

Primary Factor: 
1: Identify Content 

Objectives Ed<(Bus+Gov)** Ed<Bus** Ed<Gov** LEA>SEA** LEA>(SEA+NRD)* 

2: Design Learning 

Experiences " LEA<(SEA+NRD)** SEA<NRD** 

3: Communicate and 

Gather Information Ed>lBus+Gov)** Ed>Bus** Ed>Gov** LEA>SEA** LEA>(SEA+NRD)** 

4: Use Systematic Means , 
for Identifying 

JobContent Ed>(Bus+Gov)* , Ed>Gov* LEA<SEA** LEA<(SEA+NRD)** SEA>NRD* 

5: Evaluate Curriculum 

Effectiveness LEA>SEA** LEA>(SEA+NRD)** 

8: Maintain Awareness of 
Relevant Developments 

Elsewhere LEA<SEA** LEA<(SEA+NRDr 

NOTE: Comparisons performed only on factors exhibiting significant one-way analysis of variance. The general factor was significant at level p<.01; all 

primary factors listed were significant at level p<.0001 . Comparisons performed using Dunn's test (Kirk, 1968). Double ellipses ( ) represent 

nonsignificant comparison; blank space indicates comparison not performed. 

^ Abbreviations Ed, Bus, and Gov stand for education, business/industry, and government, respectively; LEA stands for local education agencies, 
SEA for state education agencies, and NRD for national R&D labs, education groups whose composition is discussed in the text. 

* p<.05. 



Primary Factor 2: Design Learning Experiences. 



Primary Factor 2 was found to deal with the teaching-learning interface and with systematic 
attempts to facilitate student learning. The emphasis appears to be on matching students, teachers, 
and situation characteristics to ensure optimal learning and information/skill transferral. Thus, one 
of the foci is upon students; the curriculum developer should design curriculum appropriate to the 
general level of student capabilities and determine appropriate situations for them to exhibit acquired 
skills and knowledges. Another focus is upon instructors; the curriculum developer should have knowl- 
edge of instructor capabilities so that instructors can be assigned appropriately to types of teaching 
situations, or in-service training of instructorc can be provided where necessary. A third focus is on 
the teacher-learner interface itself. This focus encompasses teacher-student role relationships and the 
design of alternative teaching strategies for different groups of students. 

There were no significant differences among the three primary sampling units— education, busi- 
ness, and government— on this factor. However, within the education group, it was clear that the 
national R&D lab subgroup scored significantly higher than the remaining eight subgroups. In fact, 
the national R&D lab group scored significantly higher on this factor than the state education agency 
group. The difference between the local education agency and the state education agency groups was 
not significant; thus, the major difference on this factor was between the national R&D lab group and 
all other curriculum developers. 

In general one might expect the national R&D lab group to generate knowledge about optimal 
teaching-learning outcomes, and those in the other groups primarily to utilize such knowledge in the 
design of curricula. The national R&D lab group might therefore be expected to score higher on this 
factor since the activities which constitute the factor may form the greater part of their research ef- 
forts. Accordingly, these activities would be less critical (present less of a problem) to the other 
groups of curriculum developers, as they would rely on the expert judgment and experimental results 
of the national R&D lab group. 

Primary Factor 3: Communicate and Gather Information . 

Primary Factor 3 appears to isolate tasks which involve exchange of information for the improve- 
ment of curriculum. According to this factor, a part of the information exchange consists of informa- 
tion outflow to the community, to instructors in terms of in-service training, and to funding agencies 
to obtain funds for curriculum development. Another part of the information exchange involves the 
gathering of information , accomplished by surveying employers to identify needed skills and knowl- 
edges and possible near-future trends, and surveying instructors to identify their needs for curriculum 
materials. The overriding concern expressed by the activities in this factor is the improvement of cur- 
riculum through the exchange of information. 

Since the activities contained in this factor appeared to relate more to applied settings, i.e., to 
involve the sorts of activities which persons working with actual curricula might perform, it was ex- 
pected that the local agency subgroups would score higher than others or this factor. The LEA group 
did, in fact, score significantly higher on this factor than both the SEA and SEA-NRD groups. The 
business and government groups, which appeared to have a more applied orientation than the educa- 
tion group as a whole, scored significantly lower on this factor than the education group. Explana- 
tions for this could include the following: First, the organizations in which curriculum developers 
in business and government are employed may employ other persons to perform the communication 
functions constituting this factor. Second, since curriculum developers in business and government 
are more in contact with the world of work (i.e., employers, employment situations, etc.), the 
activities making up this factor may present less of a problem to them. Third, curriculum developers 
in business include both those involved in company training programs and those developing programs 
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for public use (e,g,, proprietary schools). The former would not be expected to be as concerned as 
the latter with the activities contained in this factor; thus, the business group as a whole would score 
lower. Support is given to the first and third explanations by the data showing that lower percentages 
of curriculum developers from business and government than from public education reported per- 
forming these activities. 

Primary Factor 4: Use Systematic Means for Identifying Job Content. 

Various strategies of job analysis appear to constitute Primary Factor 4. The major focus of the 
factor is upon obtaining worker performance information. One method of obtaining such information 
is through the use of task inventories to survey employees about their job tasks. Alternatively, job 
analysts may be employed to visit job sites to record information on worker performance and to sur- 
vey employers about needed job skills and knowledges. In either case, the information is used to 
formulate student performance objectives. The major theme of the items which loaded on this fac- 
tor is the empirical determination of curriculum content through the use of various job analysis tech- 
niques. 

The education group scored significantly higher on this factor than the government group, but 
the differences between the education and business groups, and between the business and government 
groups, were small and nonsignificant. The overall picture was one of relative similarity among the 
three large groups. However, within the education group, there were marked differences among the 
subgroups. The education subgroup which scored higher on this factor was the SEA group. The na- 
tional R&D lab group scored much below the SEA group; their score more nearly approximated that 
of the LEA group, indicating a relatively lower level of rated criticality on the factor. This suggested 
that the SEA group was more concerned with job analysis and, as with Primary Factor 2, the local 
agency subgroups may have seen such activities as less critical, relying instead on the results generated 
by the state-level groups. 

Primary Factor 5: Evaluate Curriculum Effectiveness. 

Primary Factor 5 appears to consist mainly of curriculum evaluation activities. One method of 
evaluation contained in the factor is the surveying of both former students and employers of former 
students in an attempt to assess the effectiveness of curriculum. Other curriculum evaluation issues 
deal with the comparison of actual versus intended learning outcomes, and the evaluation of existing 
curriculum content for the intended purpose of learning. A small part of this factor consists of the 
use of standard tests and other survey methods to identify individual student and group character- 
istics. Evaluation, using various methods to assess various outcomes, is the central feature of the fac- 
tor. 

The assessment of group differences on this factor revealed a small but significant difference 
between the education and business groups, with the business group scoring higher. The differences 
between the education and government groups and between the business and government groups 
were not significant. Once again, there were great differences among the education subgroups. The 
LEA group scored significantly higher on this factor than the remaining, state-level subgroups (both 
the SEA and SEA-NRD groups). The factor appeared to be of an applied nature, being more impor- 
tant to those directly responsible for demonstrating curriculum effectiveness. 

Primary Factor 6: Select and Organize Materials. 

Primary Factor 6 includes all of the activities in the questionnaire under the categorical label of 
the "Selection and Organization of Materials." Included in the process of selecting and organizing 
materials are such activities as the adaptation of existing materials for local use and the preparation 
of specifications for development of materials where no existing materials suffice. Also included are 
the preparation of teaching-learning materials and the development of instructional delivery systems. 
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A small part of this factor has to do with the organization of materials designed to promote a curric- 
ulum already developed. This factor then deals primarily with the organization of materials for use 
in the curriculum. 

There were no significant differences between any of the groups on this factor. The state instruc- 
tional materials labs subgroup had the highest score, but the difference between their score and the 
scores of the remaining education subgroups was not significant. One reason for the lack of signifi- 
cant differences may have been that all groups considered selection and organization of materials to 
be less critical to their jobs than other activities, thus erasing group differences. This interpretation 
is supported by noting (Table 8) that the activities loading highest on this factor belong to category 
C (selection and organization of materials), and that none of the activities in this category was found 
to be among the most critical activities for a respondent group (see discussion of "Most Critical Act- 
ivities" and associated tables). The reason for this is suggested by separate consideration of the Im- 
portance and Problem/Need ratings given to activities in this category (i.e., activities No. 45 to No. 49). 
As shown in Table 13, a few activities were rated highly on one scale, but no activities were rated 
highly on both scales. These differences resulted in lowered criticality scores for all groups. Return- 
ing to the analysis of variance, however, the high score of the state instructional materials lab sub- 
group reflected a meaningful trend, although the small sample size prevented confirmation of a sig- 
nificant difference in the statistical testing of group differences. 

Primary Factor 7: Formulate Curriculum Policy and Plans. 

Primary Factor 7 contains activities which describe the long-term management of curriculum 
development efforts. Among these, the preparation of long-range and annual plans for curriculum 
development are key tasks. Also of major importance is the formulation of policies and priorities 
for curriculum planning and development. Determining which curricula will be developed and the 
developmental and operational costs of new curricular programs are other important tasks. Finally, 
the factor includes the use of computerized information systems to aid in the decision-making pro- 
cess. The factor is concerned overall with planning, guiding, and overseeing development of curric- 
ulum. 

This factor appears to be associated with applied work in curriculum development, and in fact, 
the local subgroups in education did tend to have higher scores. However, none of the comparisons 
among groups turned up significant differences. Again, there may have been a number of reasons for 
this, but the trend of local subgroups to score higher was consistent with the. factor's general inter- 
pretation. 

Primary Factor 8: Maintain Awareness of Relevant Developments Elsewhere. 

Primary Factor 8 appears to consist of a somewhat mixed set of activities. However, concern 
for the integration and adaptation of the research and support of other groups is evident. One aspect 
of the factor involves the determination of where and with what success the same curriculum prob- 
lems are being worked on elsewhere. Another aspect ir. the relation of curriculum information and 
materials developed elsewhere to the local curriculum. A final aspect of the factor includes coopera- 
tion with other agencies and the use of advisory panels to assist in curriculum formulation. This fac- 
tor then involves the integration of curriculum Information from other agencies, research organiza- 
tions, and advisory panels in order to benefit from existing knowledge and research in the field, and 
the adaptation of this knowledge and materials to the local curriculum and to local problems. 

There were no significant differences between the three major groups of curriculum developers, 
but there were marked differences among the education subgroups. The SEA group scored significantly 
higher than the LEA group on this factor; the SEA-NRD group was also significantly higher on this 
factor than the LEA group, but at a lower level of significance. In general, the subgroups constituting 
the SEA group rated the activities on this factor as more critical to their present jobs than the LEA 
group, again suggesting that the local agency subgroups may have been relying on the state-level sub- 
groups for state-of-the-art guidelines for developing actua! curricula. 
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CONCLUSIONS AND IMPLICATIONS 



Background data on respondents showed that those in public education and government 
resembled each other with regard to a somewhat lower level of involvement with general education 
than with vocational education audiences. Concerning the extent of their involvement in curriculum 
development activity, the latter two groups were also similar, devoting approximately one-half of their 
time to curriculum development. Those in business averaged one-third of their time in curriculum 
development. However, all three groups reported a very similar division of their curriculum develop- 
ment efforts among the several categories of activity defined in the questionnaire. 

The content of the curriculum development activity list was validated by respondent ratings 
which reported performance of all the activities by substantial percentages of the respondents. Thus, 
the inventory of activities can serve as a valuable resource for a competency based approach to the 
preparation of curriculum developers. 

Criticality scores computed for each of the curriculum development activities contained in the 
questionnaire provided a summary indication of each activity's importance and the degree of problem 
encountered in performance. The more critical activities overall tended to be in the areas of curric- 
ulum management and administration, curriculum content selection and organization, and curriculum 
evaluation. An appreciable number of specific activities in these areas were found to be highly critical. 
Conversely, activities involving curriculum materials selection and organization and the design of in- 
structional plans and strategies were found to be generally less critical. 

Factor analysis of the curriculum development activities yielded eight primary factors and one 
general factor. Each of the eight primary factors was characterized by activities from a single apriori 
category, thus enhancing the validity of the initial hypothesis that the total list of activities represented 
a number of interpretable clusters. A summary of the results of the factor analysis would have to focus 
on the different patterns of critical activities for some subgroups of curriculum developers and the 
relative lack of differences between other subgroups. 

Considering first the similarities or lack of differences in the data, the two composite groups 
identified as business and government did not differ significantly on any of the factors. Curriculum 
developers in business and government tend^ to hold similar views of the activities in the inventory, 
and performed somewhat similar jobs. Thus, if procedures were developed and made available to aid 
performance of any of the activities, curriculum developers in business and government would tend 
to benefit equally. 

Another similarity evident in the data was the absence of differences among thG three major 
respondent groups— education, business, and government— on four of the primary factors. Thus, im- 
provement of procedures to perform activities in Factors 2, 6, 7, and 8 would also tend to benefit 
all curriculum developers to about the same extent. 

As to differences between the three major groups, clear group differences were evident on three 
factors. The general curriculum development factor (I) and Primary Factor 3, dealing with informa- 
tion- exchange for curriculum improvement, showed significantly higher scores for the curriculum 
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developers in education. Primary Factor 1, dealing with selection and organization of content, re- 
vealed higher scores for the business and government groups. 

A final set of differences evident in the data were those between education subgroups. The 
nine education subgroups separated rather clearly into two major subgroups termed the local educa- 
tion agency and state education agency subgroups, respectively. The subgroups central to the local 
agency category were the curriculum developers in technical institutes, area vocational schools, local 
school districts, and community and junior colleges. The subgroups making up the state agency group 
were those employed in Vocational Education departments in colleges and universities. State Divisions 
of Vocational Education, state instructional materials labs, and RCU's. The national R&D lab sub- 
group had a profile of scores on the factors which did not clearly resemble the profiles of either the 
local or the state groups. 

The local agency subgroups scored higher than the state subgroups on three primary factors. 
Factors 1, 3, and 5. These factors, dealing with the selection and organization of content, information 
exchange for curriculum improvement, and evaluation, appear to encompass tasks which were more 
applicable at the school level, tasks which face curriculum developers in their attempts to set up mean- 
ingful curricula and to make sure their curricula are accomplishing their purposes. 

On the other hand, the state education agency subgroups scored higher than the local education 
agency subgroups on Primary Factors 2, 4, and 8. These factors concerned the design of teaching and 
learning strategies, the use of job analysis in the derivation of curriculum content, and the integration 
and adaptation of other curriculum development efforts. These factors represented activities which 
appeared to be more appropriate for agencies whose accomplishments would have wider dissemination 
than those of local agencies. Thus, state level institutions might be expected to disseminate the results 
of the above activities to the local level for use in development of actual curricula. The national R&D 
lab subgroup, while scoring below average on eight factors, scored extremely high on Primary Factor 
2, the design of teaching and learning strategies. This result implied that the sample constituting the 
national R&D lab respondents had a rather special focus, teaching and learning strategies, and were 
less similar to other non-local subgroups than may at first have been expected. 

In summary, developments in the areas of curriculum content selection, curriculum improve- 
ment through information exchange, and curriculum evaluation (Primary Factors 1, 3, and 5) would 
benefit curriculum developers working in the local agency context to a greater extent than other 
groups. On the other hand, advances in teaching and learning strategies, job analysis, and the inte- 
gration and adaptation of other efforts (Primary Factors 2, 4, and 8) would directly benefit those in 
the state agencies more than any others; however, the resultant information generated by the state 
level groups should be of great benefit to all local agency curriculum developers. Thus, different 
groups had different priorities, and saw different types of activities as more important to their jobs. 
Although the state and local subgroups would benefit most from different types of aid, each group 
appeared to have considered all tasks as relatively critical. Therefore, significant advances in any of 
these areas would be valuable to all persons identified as curriculum developers. 
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Respondent Ratings of Curriculum Activities 

Survey respondents were asked to provide ratings of curriculum activities in two dimensions. 

The first was to provide a measure of the problem or need involved with respect to guidebooks, 
manuals, training, etc. The rating scale used was as follows: 

0 = I do not do this activity. 

1 = Little if any problem; no help needed. 

2 = A moderate problem; help would be useful. 

3 = A major problem; need all the help I can get. 

In addition to providing answers to the question of Problem/Need, zero and non-zero responses ob- 
tained distinguished non-performers from performers. 

The second question concerned the perceived importance of each activity to the respondent's 



Here the scale used was: 


1 = 


Not important 


2 = 


Slightly important 


3 = 


Moderately important 


4 = 


Very important 


5 = 


Most important 



Summary ratings given to each activity, for both Problem/Need and Importance, are presented 
in Table 10. In addition, percentages performing each of the activities is shown. Activities are ar- 
ranged in accordance with the activity categories employed in the survey questionnaire. Summary 
responses are shown separately for the three major respondent groups, keyed to column headings 
as follows: 

a: Respondents in public education (n = 260) 
b: Respondents in business/industry (n = 41 ) 
c: Respondents in government = 27) 
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Table 10 

Respondent Ratings of Curriculum Activities 



Activity Percent Performing Importance Problem/Need 



Number 


Activity 


a 


h 


c 


□ 


b 


p 


□ 


u 


c 




CATEGORY A: CURRICULUM MANAGEMENT AND 
ADMINISTRATION 




















1 


Conduct staff in-service training for curriculum revision efforts 


83,0 


68.3 


59.3 


4.06 


3.36 


3.22 


1.58 


0.98 


0.96 


2 


Write proposals to obtain funds for curriculum development 
projects 


73.1 


48.8 


38.5 


3.54 


2.65 


3.i6 


1.40 


0.78 


0.46 


3 


Survey instructors formsliy to identify their needs for 
curriculum/instructional materials 


80.4 


78.0 


55.6 


3.73 


3.50 


3.23 


1.27 


1.10 


0.93 


4 

u 

^ 5 


Survey employers to identify skills and knowledge needed 
by students for entering employment 


68.9 


57.5^ 


51.9 


4.09 


3.65 


3.41 


1.34 


1.00 


0.74 


Determine which curriculum will be developed 


87,3 


85.4 


81.5 


4.03 


3.76 


3.65 


1.57 


1.39 


1.26 


6 


Determine the attitudes to be developed in a curriculum 


82,6 


87.5 


74.1 


3.90 


3.54 


3.15 


1.64 


1.53 


1.37 


7 


Consult with employers to identify near-future trends and 
devf ioDTients in occupations as they relate to curricula 


69.9 


68,3 


61.5 


4.04 


3,85 


3.12 


1.37 


1.24 


1.08 


8 


Survoy local education agencies to identify their needs for 
curriculum and instructional materials 


61,4 


29,3 


19.2 


3.31 


2,21 


1.96 


1.02 


0.39 


0.23 


9 


Obtain and analyze survey data (e.g., census, community needs, 
company needs) to help establish curriculum policy and ' 
development priorities 




CO R 




0.34 


o.UO 




1 Ai 


1 nn 


1 OA 


10. 


Provide information to the community about the nature and 
content of curriculum 


82,2 


60.0 


40.7 


3.78 


2.97 


2.65 


1.42 


1.00 


0.67 


11 


Supervise design of exhibits and displays to promote 
developed curriculum and instructional materials 


62.0 


48.8 


40.7 


2.91 


2.38 


2.35 


1,00 


0.78 


0.56 


12 


Write materials (e.g.. brochures, journal articles) to promote 
developed curriculum and instructional materials 


77.2 


57.5 


70.4 


3.39 


2.87 


2.81 


1.36 


0.98 


1.11 




13 Dmibe and promote curriculum activities and products at 
local, state, regionaUr national professional meetings 68.1 46.3 44.4 3.12 2.45 2.35 1 12 071 059 

14 Work with local personnel to modify and develop support 
materials for local use of curriculum and instructional 

83.8 75.6 63.0 3.79 2,95 3.42 1.52 1,24 1.15 

15 Determine the developmental and operational costs of new 
curricula/instructional programs 81.9 87.5 66.7 3.74 3.46 3.46 156 1.18 1.26 

16 Determine where the same curriculum problems are being 
worked on elsewhere 84,6 72.5 70.4 3.74 3.18 3.52 1.75 1.18 1.15 

17 Define curriculum development goals or objectives in 
^P^^2^'°"^l^^^"^s 91.9 85.0 85.2 4.00 3.67 3.89 1.64 1.40 1.41 

18 Establish job requirements for personnel who are going 
to develop curriculum 66.2 63.4 44.4 3.36 2.76 3.07 1.09 0.31 0.59 

19 Relate curriculum information derived elsewhere to a 
'^^^'P^^^'^"^ 90.7 71.8 70.4 3.49 3.05 3.22 1.56 1.13 0.96 

^ 20 Use computerized information systems to obtain data for 

decision-making 52.3 29.3 37.0 3,05 2.23 2.50 0.94 0.56 0.59 



00 



21 Identify new methods and procedures for developing curriculum 
and instructional materials 

22 Cooperate with other agencies to jointly develop curriculum 
and instructional materials 



87.6 80.5 81.5 3.80 3.60 3.70 1.79 1.44 1.56 

80.3 61.0 70.4 3.72 2.98 3.70 1.51 0.93 1.15 

23 Formulate policies and pfiorities for curriculum planning 

and development 86.0 P0.2 77.8 3.88 3.29 3.81 1.60 1.29 1.30 

24 Prepare a long-range plan for curriculum/instructional 

program development B4.9 82.9 55.6 4.08 3.27 3.85 1.69 1.37 1.00 

25 Prepare an annual plan for curriculum/instructional 

program development 84.9 75.6 55.6 3.91 3.08 3.22 1.56 1.12 0.93 

26 Supervise curriculum and instructional materials 

development work 82.6 85.4 74.1 3.80 3.51 3.56 1.43 1.22 1.11 

a: Respondents in public education 
b: Respondents in business/industry 

Q c; Respondents in government . ^ 
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Table 10 ■ Continued 



Activity ^ Percent Performing Importance Problem/Need 

Number ^'^''^'^Y a b c a b c a b c 



CATEGORY B; SELECTION AND ORGANIZATION 
OF CONTENT 



27 


Survey employees in business/industry about their job tasks 


50.4 


65.9 


15,4 


3,68 


3.55 


2,69 


0.93 


1.17 


0.31 


28 


Utilize experienced instructors to identify curriculum content 
and to write instructional materials 


86.5 


85.4 


69,2 


3,98 


3.93 


3.48 


1.47 


1.29 


1.04 


29 


Prepare occupational task inventories for use in curriculum 
development 


59.9 


56,1 


33,3 


3,61 


3.20 


2,81 


1.15 


0.93 


0.59 


30 


Employ trained job analysts to obtain information on vi/orker 
performance for use in curriculum development 


26.0 


24,4 


12,0 


2.84 


2.38 


2,85 


0.46 


0.41 


0.24 


31 


Search published textbooks as a primary source of content 
for curriculum 


70.7 


73,2 


65,4 


2.86 


3.28 


2.67 


1.01 


1.20 


1.04 


32 


Visit job sites to record observations of worker performance 
for use in curriculum development 


41.9 


68.3 


48,1 


3.33 


3.24 


2.85 


0.67 


0.95 


0.67 


33 


Utilize advisory committees to assist in curriculum 
formulation 


86.9 


53,7 


60,0 


4.15 


3.05 


2.92 


1.40 


C.68 


0.80 


34 


Determine the performance objectives to be included in 
a curriculum 


84.1 


f\f\ 0 

90,2 


85,2 


4.10 


4.05 


i AJ 

4.04 


1.53 


4 A A 

1.44 


1.59 


35 


Determine the knowledges/concepts to be included in 
a curriculum 


85.3 


87,5 


92,6 


4.15 


4.00 


4.15 


1.54 


1.48 


1.63 


36 


Identify prerequisite and enabling skills and knowledges 
for each curriculum or instructional unit 


80.3 


85,4 


66.7 


3.86 


3.66 


3.78 


1.43 


1.32 


1.15 


37 


Select curriculum content as intended learning outcomes 


81.8 


82,9 


61.5 


3.83 


3.60 


3.50 


1.44 


1.34 


1.15 


38 


Sequence the series of intended learning outcome' 


m 


82,9 


74.1 


3.72 


3.45 


3.37 


1.37 


1.34 


1.15 


39 


State the ultimate performance objectives 


80.7 


90,2 


84,6 


4.12 


<i.00 


4.15 


1.47 


1.29 


1.50 


40 


State the requisite enabling objectives (behavioral 
requirements: knowledge, skills, attitudes) 


80.7 


85.4 


76.9 


3.99 


3.70 


3.63 


1.47 


1.27 


1.27 



41 Identify potential units of instruction 82.7 85.0 73.1 3.75 ' 3.49 3,15 1,37 1,33 112 

42 Determine the content and success of new curriculum 

developed elsewhere 81.5 62,5 74.1 3.61 3.00 3,15 1.52 1.03 1.33 

43 Determine the extent that proposed curricula have gone 

through experimentation or validation 78,4 65.9 59.3 3.58 3,21 2,89 1.44 1,2? 1,07 

44 Prepare statements of student performance objectives from 

occupational task performance data 63.5 63,4 40.7 3.66 3.33 2.81 1.18 0.98 0.78 

CATEGORY C: SELECTION AND ORGANIZATION 
OF MATERIALS 

45 Adapt available curriculum and instructional materials to 

meet local curriculum development needs 82,7 78.0 81,5 3.87 3,59 3,59 1.47 1.20 119 

46 Prepare specifications for development of necessary 

curriculum materials and instructional aids 74.2 76.9 66.7 3.62 3,46 3.38 1,36 1.21 1.07 

47 Prepare teaching-learning materials (e.g., transparencies, 

information sheets, models, slides) 58.1 78.0 55.6 3.64 3.63 333 0.99 1.32 0.85 

u 

(Ji 48 Describe the physical facilities and list necessary equipment, 

tools, and/or materials as part of a curriculum plan 70.8 82,9 66.7 3.62 3.34 3.26 1.22 1.12 096 

49 Develop total instructional delivery systems (e.g., instruc- 
tional packages, audiovisual materials, etc.) 60.0 63.4 53.8 3.79 3.41 3.56 1.17 0.98 1.00 

CATEGORY D: DESIGN OF INSTRUCTIONAL PLAN AND 
ALTERNATIVE TEACHING STRATEGIES 

50 Survey students to identify individual and group characteristics 50.8 68,3 33.3 3,53 3.12 2.92 0.92 1.00 0.56 

51 Use standard tests to identify individual and group 

characteristics 44,2 41,5 25.9 3.11 2.46 2.77 0.73 0.63 0.30 

52 Assess the adequacy of existing curriculum and instructional 

materials for use in local curriculum development 83.3 78,0 88.9 3.77 3.46 3.56 1.52 1.24 1 33 

53 Derive instructional strategies and learning experiences to 

support specific learning outcomes 74,2 73,2 66.7 3,77 3.53 3.41 1.42 1,29 1.22 

a: Respondents in public education 
^(■j b; Respondents in business/industry 

ErJc ^^P°"'^^"^5 in government 50 
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Activity Percent Performing Importance Problem/Need 

Number ^'^^^ a b c a b c a b c 

CATEGORY D: DESIGN OF INSTRUCTIONAL PLAN 
AND ALTERNATIVE TEACHING 
STRATEGIES (Continued) 

54 Recommend instructional techniques and learning experiences 

as part of a curriculum plan 84.2 87.8 77.8 3.87 3.80 3.41 1.53 1.34 U3 

55 Describe instructor planning activities in preparing for presen- 
tation of a curriculum or instructional unit 68.1 75.6 63.0 3.55 3.44 2.93 1.19 1.07 1.04 

56 Design alternative instructional treatments (i.e., materials 

and experiences) for variously defined subgroups of students 60,0 68.3 59.3 3.60 2.95 2.85 1.19 1.10 1.04 

57 Create situations by which students can exhibit intended 

learning outcomes 65.0 75.6 51.9 3.71 3.30 3.33 1.21 1.24 1.04 

58 Determine new role relationships between instructors and 

pupils required by new curricula 59.9 53.8 59.3 3,44' 2.72 3.00 1.17 0.87 0.93 

59 Determine how demanding new curricula will be of students 72.1 85.4 44.7 3.48 3.44 3.22 1.32 1.32 0,96 

60 Determine how new curricula are going to require instructors 

to change or be retrained 83.5 78.0 63.0 4.01 3.33 3.22 1.64 1.15 1.04 

61 Identify alternative methods of instruction (including major 

resources and types of instructional aids) 81.5 85.4 81.5\^ 3.80 3.40 3.33 1.60 1.44 1.37 

CATEGORY E: EVALUATION OF CURRICULUM 

62 Evaluate the utility of student performance and enabling 

objectives 76.5 80.0 70.4 3.94 3.60 3.44 1.49 1.48 1.15 

63 Field test and revise instructional materials prior to 

publication 60.2 56.1 40.7 3.64 3.32 3.04 1.13 0.98 0.70 

64 Evaluate the appropriateness of existing curriculum content 

for the intended purpose of learning 83.8 92.7 92.6 4.07 4.05 3.96 i.6i 1.56 1.48 



65 Survey former students to assess curriculum effectiveness 65.5 78,0 55.6 IOl 3.90 3.67 1 26 141 089 
6C Survey employers of former students to assess curriculum 

60.5 63,4 51,9 4.01 3.85 3.44 1,14 1.10 0.85 

67 Compare actual learning outcomes with intended learning 

76.2 78,0 55.6 4.09 3,88 3.89 1.50 1.41 1.07 

68 Revise curriculum and instructional materials based on 

^'^'^'"'^^^^ 77.3 73.2 63.0 4.06 3.98 3,59 1,48 1,22 1.11 



a: Respondents in public education 
b: Respondents in business/industry 
c: Respondents in government 
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Summary Analyses of Impm unce 
and Probiem/Need Ratings 



Appendix B lists all of .he curricul'.im activities for which ratings of Importance and Problem/ 
Need were obtained. These ratings are the components of the "criticality" scores which are discussed 
and analyzed in the body oi the report. It Ij of some interest, as well, to know which activities were 
given the highest component ratings, and by whom. This information is shown in Tables 1 1 and 1 2 
for the three principal respondent groups, and overall. Similarities and differences between the rank- 
ings of these groups art indicated. 

For example, it may be observed in Table 1 1 that the five most important activities in overall 
ranking (No, 35, 39, 34, 64, and 67) are also anong the top ten for each group individually, and that 
the top four overall are identical to the top four for government and business. And, despite the in- 
fluence of the large size of the public education group on the overall rankings, the "most important" 
activity for this dominant group (No. 33) does not even appear among the ten most importa; over- 
all 

The five activities highly rated for Importance by the total respondent group, as identified in 
Appendix A, are as follows: 

No. 35 Determine the knowledge/concepts to be included in a curriculum. 

No. 39 State the ultimate performance objectives. 

No. 34 Determine the performance objectives to be included in a curriculum. 

No. 64 Evaluate the appropriateness of existing curriculum content for the intended purpose 
of learning. 

No. 67 Compare actual learning outcomes with intended learning outcomes. 

Activities rated very highly by a particular group, but not among the top five overall were: 

No. 33 (Public Educ-r^tion) Utilize advisory committees to assist in curriculum formulation. 

No. 68 (Business/Industry) Revise curriculum and instruction materials based on field test 
data. 

No. 17 (Government) Define curriculum development goals or objectives in operational terms. 

There was less consensus among the three groups regarding activities presenting the greatest Prob- 
lem/Need, as indicated in Table 12. However, among the ten highest ranked activities for the respon- 
dent group overall are also five which appear in the lists of each of the individual groups. These are: 
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No. 21 



Identify new methods and procedures for developing curriculum and instructional 
materials. 



No. 6 Determine the attitudes tc be developed in a curriculum. 

No. 17 Define curriculum development goals or objectives in operational terms. 

No. 61 Identify alternative methods of instruction (including major resources and types of 
instructional aids). 

No. 64 Evaluate the appropriateness of existing curriculum content for the intended purpose 
of learning. 

Because of the special interest of this survey in tha concerns of curriculum developers in public 
education, and especially in vocational education, an additional analysis was made. Responses of the 
public educatio 1 -^roup were analyzed in accordance with the sample strata shown in Table 1 of the 
following suby: oups: 

1. National R&D laboratories and state instructional mater luls laboratories. 

2. Community and junior colleges. 

3. Technical institutes and area vocational schools. 

4. Departments of vocational education in colleges and universities. 

5. State divisions of vocational education and research coordinating units. 

6. Local school districts. 

Activities most highly rated by these subgroups as to Importance and Problem/Need are identified in 
Table 13. The ten to twelve activities receiving the highest rating for each subgroup are included in 
the lists, identified numerically and arranged in numerical order, rather than by rank order of sig- 
nificance. In this way it becomes more apparent, not only that the most significant activities differ 
greatly from one subgroup to another, but that the distribution of these activities within the several 
activity categories is also very different. For example, the most significant Problem activities for the 
community and junior college respondents were all in the curriculum management and administration 
area (i.e.. Problem activities are all numbered between 1 and 26). In contrast. Problem activities of 
R&D labs and instructional materials labs were found in all of the activity categories. Several other 
facts of interest may be noted as well: 

1. There is no activity, either in the lists of Importance or Problem activities, which is 
common to all six subgroups. 

2. Activities which are highly rated both as to Importance and Problem by any given 
group are few in number and are rather unique to the group. 

3. In the individual lists for each group, curriculum management activities occur most 
often, while curriculum content activities are in second place; the remaining cate- 
gories rank differently in the Importance and Problem lists. 
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With regard to the second observation above, activities rated most highly both for Importance 
and Problem/Need are as follows, for each subgroup: 

National R&D Laboratories and State Instructional Materials Labs 

No. 21 Identify new methods and procedures for developing curriculum and instructional 
materials. 

No. 34 Determine the performance objectives to be included in a curriculum. 

No, 35 Determine the knowledges/concepts to be included in a curriculum. 

No. 63 Field test and revise instructional materials prior to publication. 

Community and Junior Colleges 

No. 4 Survey employers to identify skills and knowledge needed by students for entering 
employment. 

No. 5 Determine which curriculum will be developed. 

No. 7 Consult with employers to identify near-future trends and developments in occupa- 
tions as they relate to curricula. 

No. 15 Determine the developmental and operational costs of new curricula/instructional 
programs. 

Technical Institutes and Area Vocational Schools 

No. 1 Conduct staff in-service training for curriculum revision efforts. 

No. 7 Consult with employers to identify near-future trends and developments in occupa- 
tions as they relate to curricula. 

No. 9 Obtain and analyze survey data (e.g., census, community needs, company needs) to 
help establish curriculum policy and development priorities. 

Vocational Education Departments in Colleges and Universities 

No. 6 Determine the attitudes to be developed in a curriculum. 

No. 35 Determine the knowledges/concepts to be included in a curriculum. 
State Divisions of Vocational Education and Research Coordinating Units 

No. 17 Define curriculum development goals or objectives in operational terms. 

No. 62 Evaluate the utility of student performance and enabling objectives. 

No. 64 Evaluate the appropriateness of existing curriculum content for the intended purpose 
of learning. 
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Local School Districts 

No. 1 Conduct staff in-service training for curriculum revision efforts. 

No. 24 Prepare a long range plan for curriculum/instructional program development. 

No. 52 Assess the adequacy of existing curriculum and instructional materials for use in 
local curriculum development. 

No. 60 Determine how new curricula are going to require instructors to change or be re- 
trained. 

No. 64 Evaluate the appropriateness of existing curriculum content for the intended purpose 
of learning. 
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Table 1 1 



Ten Most Important Activities of 
Each Respondent Group 



Respondent Group and Activity Number 



Activity Government Business/ Public Overall 




* Among the ten highest rated activities of each respondent group. 
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Table 12 



Ten Activities Which Involve the Greatest 
Problem /Need for Each Respondent Group 



Respondent Group and Activity Number 

Government Business/ Public Overall 

A^^'^'^Y (n = 27) Industry Education (n = 328) 

^^^^ (n = 41) (n = 260) 




* Among the ten highest rated activities of each respondent group. 
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Table 13 
Activities Most Highly Rated by 
Respondent Subgroups within Public Education 



RAD and 
Instr. Mat 'Is 
Labs 


V^fJI 1 II MU 1 II Ly 

& Junior 
Cnllpap^ 


1 cUi It 1 lUdi 

Institutes & Area 

\/nratinnal 

Schools 


oui icyc 

Dep'ts of 
Voc-Ed 


O La Lc 

Voc-Ed 

r)iwi<iinn<i 

& RCU's 


Local 
School 
nistrirt^ 

L</ U LI 1 ^ 


8 


4 


1 


1 


4 


1 


21 


5 


4 


6 


7 


5 


26 


7 


7 


24 


17 


24 


34 


15 


9 


34 


33 


35 


35 


33 


28 


35 


34 


39 


36 


39 


33 


40 


35 


45 


39 


65 


34 


64 


39 . 


52 


49 


66 


35 


65 


62 


60 


63 


67 


65 


66 


64 


64 


68 


68 


66 


68 


67 


67 




67 




68 





Problem/' 

Need 

Activities 



6 


4 


1 


2 


16 


1 


16 


5 


5 


6 


17 


17 


21 


7 


6 


16 


21 


21 


22 


9 


7 


21 


22 


24 


23 


10 


9 


22 


24 


34 


34 


15 


15 


35 


43 


52 


35 


16 


16 


42 


60 


54 


41 


17 


19 


43 


61 


60 


43 


21 


24 


60 


62 


61 


46 


24 


39 


61 


64 


64 


53 


25 


52 








63 













Importance 
Activities 



Activities are identified by number (see Appendix A), and are listed in numerical (not rank) order. Activity 
numbers within a box belong to the same activity category. 
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APPENDIX C 

Analysis Methodology 




Analysis Methodolopy 



ProblefTi/Need scores were averaged using the full scale of responses (0-3) rather than deleting 
the responses ot non-p?rformers (zeros). Reasons for using the entire scale in computing mean values 
were: 

1. Once the target population and sample were determined, they became, by definition, the 
populc tion of interest. To cah-;ulate Problem/Need on the basis of performers only would 
Ire e luivalent to redefining the target group for each activity rated. 

2. None of the activities were undertaken by all of the respondents. Averaging for performers 
only would indeed focus attention on some activities which were a great problem, but 
would distort the needs of the group as a whole. 

3. Scores for performers only can be readily computed from the data in Appendix A for the 
three main respondent groups. The mean score for performers only is obtained by dividing 
the mean score for all respondents, as given in Appendix A, by the proportion of persons 
performing the task. For example, the mean Problem/Need rating for Activity 1 for cur- 
riculum developers in public education was 1,58. Since only 83% of the respondents in 
public education reported performing the activity, the mean rating for public education 
performers only is 1 .58 divided by .83, or 1 .90. 

4. The Problem/Need scale is a complex continuum composed of the elements "performance" 
and "degree of problem/need." With this in mind, the "0" point on the scale becomes a 
rational zero point on the dimension. 

5. From a measurement and analysis point of view, since using data from performers only 
would in effect redefine the sample for each task, there would be no basis for comparison 
of mean values across tasks. In addition, computation of criticality scores would not be 
possible. While this might seem to be an argument against use of criticality scores, a factor 
analysis of the Problem/Need scale alone would also be impossible if data from performers 
only were used. 

o. Finally, the aim of the analysis was to calculate the Problem/Need value for each activity 
for the average curriculum developer across all institutions sampled. Because of this and 
because the data of the study reflected differences in how people verbalized their problems 
and needs, reference back to the Problem/Need scale values for interpretation was some- 
what problematic. No criterion measures were gathered in the study (and compelling 
criterion measures would be hard to design for the present case), so there was no basis for 
designation of any point on the scale as the threshold of need, separating activities requir- 
ing aid from those that do not. However, the scale means calculated across all respondents 
do allow a rank ordering of activities, reflecting the relative magnitude of Problem/Need 
for the average curriculum developer. 
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The criticality rating scores were formed in the following manner. First, since Problem/Need 
and Importance were rated using different scale ranges (O-to-3 and 1-to-5 respectively) which dif- 
fered as to the number of scale intervals (three and four, respectively), the Problem/Need ratings 
were adjusted by a four-thirds factor, thus defining a new Problem/Need scale having the same n'.;m- 
ber of units as the Importance scale, and with values ranging from O-to-4. Addition of the Impo : • aiiice 
and Problem/Need ratings then resulted in this being reduced to one rating, on a l-to-9 continuum, 
for each of the sixty-eight items for each respondent. 

The criticality ratings were formed by addition, rather than some other mode of combination, 
since the model usually assumed to underlie judgments is an additive one, and because the para- 
meters of the distribution of the sum of random variables are predictable. Further, the variables, 
Problem/Need and Importance, were not separately standardized before addition. To do so would 
have involved loss of the information which is contained in the score variances, which reflect the 
degree of consensus of the judgments given, and which was considered to be important information 
to be incorporated in the criticality ratings and retained for the factor analysis. 

One 68 x 68 product-moment correlation matrix was computed from the criticality ratings on 
the sixty-eight tasks. Communalities were estimated by the squared multiple correlation of a vari- 
able with all of the remaining variables, and the principal axes factor extraction technique was used. 
Both the scree test (Cattell, 1966) and interpretability were used as criteria for the number of fac- 
tors. The factors were rotated by the varimax, binormamin, and orthoblique independent cluster 
and nonindependent cluster (Harris StKaiser, 1964) procedures. 

A second-order factor analysis was performed on the intercorrelations of the first-order factors 
following the same procedures as those outlined abo^e for the first-order analysis. A matrix trans- 
formation (Schmid & Leiman, 1957) was used to make factors on all levels orthogonal, or indepen- 
dent. All of the above procedures are standard factor analytic options (Gorsuch, 1974; Rummel, 
1970). 

Traditional notions of reliability and validity were not directly applicable to the Problem/Need 
and Importance responses. Reliability measures are usually computed to assess the stability of a 
total score over a set of items. Since the items were factor analyzed and total scores were not eval- 
uated and since the instrument was only administered one time, traditional measures could not be 
used. Therefore, since factor scores were used, factor reliabilities were computed using a generaliza- 
tion of the Kuder-Richardson method (iVIagnusson, 1966). The Kuder-Richardson riiCirhod results 
in lower bound estimates for split-half reliability; instruments, like the present one, which attempt 
to completely map a domain with minimal item overlap will tend to exhibit lower split-half reliabil- 
ities even though their stabilities, or test-retest reliabilities, may be very high. 

Finally, of the four major types of validity, the primary type which applied to the present data 
collection was Content validity; the procedures to ensure content validity are discussed in the body 
of the paper. One form of construct validity, factorial validity, is also satisfied if clear, meaningful 
factors emerge from a factor analysis. In the present study, the emergence of factors corresponding 
to the five apriori categories into which activities had been initially classified illustrated the factorial 
validity of the data and enhanced the construct validity of the hypothesized classes of curriculum 
development activities. The remaining two types of validity— predictive and concurrent— were in- 
applicable since data were not gathered on criterion measures. 
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APPENDIX D 

Most Critical Activities and Scores 
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Table 14 

Most Critical Activities and Scores, by Respondent Group 



Rank 



Public 
Education 
(n=260) 



Business/ 
Industry 
(n=41) 



Government 
(n=27) 



All 

Respondents 
(n=328) 





Activity 
Number 


Score 


Activity 
N umber 


Score 


Activity 
Numbar 


Score 


Activity 
N umbsr 


Score 


1 


24 


6.33 


64 


6.13 


35 


6.32 


35 


6.19 


2 


64 


6.21 


35 


5.97 


34 


6.16 


64 


6.18 


3 


35 


6.20 


34 


5.97 


39 


6.15 


34 


6.13 


4 


60 


6.20 


65 


5.79 


64 


5.94 


24 


6.08 


5 


17 


6.19 


67 


5.76 


21 


5.78 


17 


6.07 


6 


21 


6.19 


39 


5.73 


17 


5.76 


21 


6.07 


7 


1 


6.16 


28 


5.65 


23 


5.54 


39 


6.04 


8 


34 


6.15 


5 


5.61 


5 


5.34 


5 


6.00 


9 


5 


6.12 


68 


5.60 


52 


5.33 


67 


5.99 


10 


67 


6.10 


54 


5.59 


40 


5.33 


6 


5.93 


11 


6 


6.09 
% 


6 


5.59 


67 


5.32 


60 


5.90 


12 


39 


6.07 


62 


5.57 


36 


5.31 


68 


5.89 


13 


16 


6.07 


17 


5.55 


22 


5.23 


23 


5.84 


14 


68 


6.03 


21 


5.52 


54 


5.18 


1 


5.84 


15 


33 


6.00 


7 


5.51 


24 


5.18 


16 


5^2 


16 


23 


6.00 


36 


5.41 


45 


5.17 


40 


5.82 


17 


25 


5.99 


40 


5.40 


61 


5.16 


28 


5.82 


18 


28 


5.95 


37 


5.39 


15 


5.15 


54 


5.81 


19 


40 


5.94 


47 


5.39 


68 


5.07 


62 


5.80 


20 


61 


5.93 


41 


5.33 


37 


5.06 


61 


5.79 



67 
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APPENDIX E 

Criticality Rotated Factor Matrix 
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Table 15 



Critlcality Rotated Factor Matrix 
(Schmid-Leiman Transformation) 



Primary Factors 



Activity 






















Factor 


1 


2 


3 


4 


5 


6 


7 


8 




I 




.150 


. 1 ii* 


.J* n 


- .085 


- .091* 


- .07 


.001 


. Ill* 


.:*69 


2 




- .071 


- .012 




.067 


- .02 1 


- .006 


.065 


.11*7 


..'76 


3 


.39** 


.000 


- .006 


. 356 


- .I 'jQl 


- . 125 


.115 


- . lofl 


.ogi 


.31*2 




.390 


.(Kb 


- .Wb 




.386 


. 198 


- .119 


- .01*8 


- .082 


.501 


? 




.070 


- .001 


.326 


.OCO 


- .0^8 


- .001* 


.271 


-.11*1 


.31*2 


o 


.510 


•ai Ji 
, Jl** 


. 171 


.238 


. 101 


- .096 


- . 105 


. 1 12 


- .016 


.1*35 


7 




.OU5 


• .055 


30I* 


. 1*29 


. 158 


- .075 


- .003 


- .061* 


.528 


0 




- .220 


.OU 1 


.219 


. IT*' 


- . ii*8 


.060 


.018 


. 198 


.376 


Q 

y 


.^37 


- .030 


- .UD** 


.310 


. l.''6 


. 158 


- . loi* 


- .002 


. 180 


.^76 


10 


.^75 




. 150 


.1*87 


- . oyo 


. 106 


-.059 


.001* 


-.013 


.^11 


11 


.510 


- . 17d 


.085 


.213 


.072 


-.093 


.291* 


- .01. 1 


.039 


.lil*8 


u* 


. 50U 


-.190 


- .036 


. 166 


-.025 


-.01*0 


.313 


.095 


.095 


.1*30 


l-'i 


liflli 


- . 301 


.105 


.11*5 


.;8i 


-.153 


.11*9 


.020 


.185 


.1*72 
.51*6 


ii* 


.50^ 


.061 


- .025 


.230 


-.071 


.065 


.090 


-.157 


.359 


15 


.1*92 


.031 


-.056 


.229 


063 


-.0I47 


.007 


.305 


.030 


.398 
.511 
.515 
.1*18 


l6 


.515 


-.063 


- .01*1* 


.066 


o6l 


-.015 


-.062 


.01*0 


.1*73 


17 


.561 


.337 


-.032 


.157 


-.051 


.017 


-.060 


.170 


.159 


.rt 


.5W 


-.01*7 


.005 


.183 


. 101 


-.098 


.060 


.218 


.111* 


19 


.551 


-.060 


- .a?H 


.1^3 


-.096 


.088 


.0I43 


-.015 


.1*11* 


.til9 
.331 
.U05 
.1*11* 


CO 


.mIiO 


-.139 


.078 


.026 


.190 


-.021* 


.027 


.272 


.005 


?l 


.513 


.058 


.030 


- .038 


.026 


-.120 


.021 


.159 


.309 


2? 


.520 


-.09? 


.008 


.121* 


.l-'?3 


-.067 


.01*8 


.010 


.311 


23 


.57? 


.050 


.oco 


- .012 


-.033 


.011* 


.055 


.510 


.025 


.595 
.712 
.712 
.503 
.558 


«!U 


.568 


.028 


-.0^7 


-.039 


-.028 


.085 


-.006 


.611 


.038 


25 


.606 


.005 




.017 


-.OCC 


.108 


.028 


.573 


.00*. 


26 


.566 


.137 


- .OPI 


.136 


- .019 


-.151 


.122 


.261* 


.110 


?7 


.503 


.081* 


-.077 


.090 


.1*80 


.213 


.060 


-.030 


-.051* 


26 


.537 


.205 


-.1.9 


.21*3 


.103 


.121* 


.001* 


-.057 


.206 


.1*76 


29 


.578 


.133 


- .OPO 


• Op3 


.1*90 


.031 


.030 


.096 


.011* 


.603 


30 


.390 




.OlB 


-.087 


.1*61 


- .01*0 


.032 


.038 


.070 


.382 


31 


.l»7y 


.211* 


• .111 


-.a?3 


-.062 


.097 


.28I4 


.061 


.059 


.388 
.1*1*14 


32 


.li6li 


.129 


-.007 


.011* 


.U17 


.110 


.100 


-.128 


.009 


33 


.^93 


.023 


-.121* 


.226 


-.027 


.11*3 


.035 


.03^; 


.21*8 


.395 
.689 
.710 
.671 
.693 
.671 


3^ 


-575 


.575 


-.001* 


.036 


.060 


- .100 


.071* 


.021 


.081 


35 


.58U 


.590 


.020 


.051* 


.067 


- .067 


.088 


.ORO 


.005 


36 


.596 


.50U 


.U?3 


.172 


.067 


-.080 


.0U8 


.000 


-.061* 


37 




.522 


.05^ 


.081 


.01*1 


-.103 


.126 


.129 


-.097 


38 


.623 


.'»55 


.L26 


.071* 


.032 


-.111* 


.175 


.060 


-.087 


39 


.590 


.595 


.095 


.033 


. 156 


-.051* 


.060 


-.c6o 


-.021* 


.71*7 
.753 
.61*6 


uo 


,6liO 


.506 




.068 


.161* 


-.065 


.Oil* 


-.01*5 


-.050 




.61*0 


'379 


.105 


. 156 


.01*7 


-.138 


.180 


.033 


-.01*1* 


u?. 


.587 


.01*6 


. 11*6 


- .005 


- .oa' 


.081 


-.067 


.018 


.31*6 


.1*99 
.581 




.591 


- .019 


..^81 


-.065 


.210 


.019 


-.155 


.027 


.280 


-.it 


.625 


.?U6 


. 128 


.085 


.319 


.005 


-.01*7 


.023 


.01*1 


.582. 


**5 


.616 


.133 


..069 


.087 


-.159 


.228 


.265 


.021 


.138 


.576 
.553 
.606 


U6 


.617 


.096 


.058 


.038 


-.052 


-.111 


.362 


.108 


.02 3 


1*7 


.53^ 


.088 


-.001 


-.187 


.065 


.065 


.516 


-.053 


-.021* 


1*8 


.586 


.170 


- . lOl* 


.090 


.065 


.lOP 


.1*10 


.027 


-.095 


.586 
.557 
.521 
.1*50 


liO 


()6l 


.087 


.0 Jl 


l3 1 


. ibo 


- »0l*5 


.1*11 


,0148 


- .010 


50 


.559 


-.112 


.251* 


"'.138 


- .071* 


.253 


.183 


-.011* 


-.098 


51 


,1*80 


-.11*8 




.085 


-.19^* 


.258 


.191 


.051 


-.055 


52 


.587 


.058 


.08i* 


.1^3 


-.227 


.236 


.031 


.110 


.177 


.521 
.566 


^3 


6^7 


.P05 


. 


- .UJ'* 


- .006 


.083 


.UCX} 


- .oi*o 


.032 


5^ 


.576 


.'15? 


.239 


.127 


-.11*2 


.019 


.112 


-.170 


.177 


.521 
.559 
.621* 


55 


.61*1* 


.172 


..?38 


.098 


.012 


.039 


.193 


-.007 


-.005 


56 


.651 


.037 


M9 


.018 


.01*6 


.01*1* 


.063 




.022 


n 


.626 


.110 


.^39 


-.061* 


.050 


.116 


.109 


-.081* 


-.071 


.61*1 


58 


.613 


.037 


.531 


.131 


.021 


-.01*2 


.009 


-.115 


-.036 


.693 


59 


.557 


.063 


.1*79 


.110 


-.070 


.103 


-.0I4I1 


.203 


-.117 


.587 


60 


.581* 


.Oli»4 


.i*l6 


.136 


-.013 


.051* 


-.206 


.071 


.081* 


.593 
.531 
.556 


61 


.571* 


.066 


.399 


.031 


-.036 


.088 


-.083 


-.057 


.119 


62 


.606 


.210 


.267 


-.021 


.121 


.166 


-.098 


-.062 


.129 


63 


.577 


.OUl 


.21*1 


-.157 


.350 


-.021* 


.092 


.0X7 


.030 


.550 


6lt 


.532 


.01*9 


.080 


-.139 


-.053 


.381* 


-.ouo 


.129 


.192 


.517 


65 


.1*78 


-.101 


.01'« 


-.01*2 


.071* 


.611* 


.038 


.063 


-.011 


.629 


66 


.1*07 


-.133 


-.011 


.076 


.11*2 


.61*9 


.021* 


-.025 


-.012 


.711* 


67 




.055 


.21*1 


-.125 


.050 


.1*32 


-.061* 


-.013 


-.013 


.570 


66 


.597 


.059 


.28J* 


-.256 


.21*7 


.177 


.091* 


.050 


.050 


.610 


Variance 




3.156 




















20.259 


2.230 


2 018 


2.0I43 


1.971 


1.583 




1.516 


36.U19 



Percent 

Common 

Variance 



55.6 



8.7 



6.1 



5.5 



5.6 



i*.3 



l*.5 



100.0 
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